Video encoding method and video decoding method

ABSTRACT

A video encoding method of encoding a multi-view image including one or more basic view images and a plurality of reference view images includes determining a pruning order of the plurality of reference view images, acquiring a plurality of residual reference view images, by pruning the plurality of reference view images based on the one or more basic view images according to the pruning order, encoding the one or more basic view images and the plurality of residual reference view images, and outputting a bitstream including encoding information of the one or more basic view images and the plurality of residual reference view images.

CROSS REFERENCE TO RELATED APPLICATION

The present application claims priority to Korean Patent Application No.10-2019-0071116 filed Jun. 14, 2019 and No. 10-2020-0070070 filed Jun.10, 2020, the entire contents of which is incorporated herein for allpurposes by this reference.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present disclosure relates to a 360-degree virtual reality (VR)image processing method. More particularly, the present disclosureprovides a method of efficiently encoding and decoding a plurality ofview images of a 360-degree VR image.

2. Description of the Related Art

Immersion may be defined as a phenomenon wherein reality and virtualreality become unclear by reproducing a virtual world which is difficultto be distinguished from reality. Immersive media includes audio, video,etc. which makes a user experience immersion. Considering that anenvironment in which media is consumed changes from PCs to smartphones,immersive media increases a sense of immersion in various VR apparatusessuch as head mounted displays (HMDs) and various types of large displayenvironments composed of multiple TVs. In particular, relatedtechnologies are being developed with the goal of providing six degreesof freedom (6DoF), such that users may experience free motion from theviewpoint of immersive video.

Conventional multi-view image compression technology considered onlycompression of a multi-view image required for a relatively smallunidirectional viewing space. However, since immersive video considers aview image required for an omnidirectionally extended viewing space, thenumber of view images increases. In addition, as the field of viewincreases, resolution also increases. Accordingly, it is necessary tosolve the problem of an increase in pixel rate reflecting input/outputinterface and the amount of data for compression processing.

SUMMARY OF THE INVENTION

The present disclosure relates to a 360-degree virtual reality (VR)image processing apparatus and method. The present disclosure provides amethod and apparatus for efficiently encoding and decoding a pluralityof view images of a 360-degree VR image. More specifically, a method ofdetermining an optimal pruning order for encoding and decoding aplurality of reference view images.

According to the present disclosure, provided is a video encoding methodof encoding a multi-view image including one or more basic view imagesand a plurality of reference view images including determining a pruningorder of the plurality of reference view images, acquiring a pluralityof residual reference view images, by pruning the plurality of referenceview images based on the one or more basic view images according to thepruning order, encoding the one or more basic view images and theplurality of residual reference view images, and outputting a bitstreamincluding encoding information of the one or more basic view images andthe plurality of residual reference view images.

According to an embodiment, the determining of the pruning order of theplurality of reference view images may include determining the pruningorder of the plurality of reference view images according to a size ofan overlapping area between a reference view image and a basic viewimage.

According to an embodiment, the size of the overlapping area between thereference view image and the basic view image may be determinedaccording to overlapping pixels between the reference view image and awarped basic view image obtained by warping the basic view image to thereference view.

According to an embodiment, the determining of the pruning order of theplurality of reference view images may include determining the pruningorder of the plurality of reference view images according to imageindices of the plurality of reference view images, and wherein each ofthe image indices may be an acquisition order of the plurality ofreference view images.

According to an embodiment, the determining of the pruning order of theplurality of reference view images may include determining the pruningorder of the plurality of reference view images according to cameraparameters of the plurality of reference view images, and the cameraparameters may include a position and/or an angle of a camera.

According to an embodiment, the determining of the pruning order of theplurality of reference view images may include acquiring, from theplurality of reference view images, a plurality of first residualreference view images, from which an overlapping area between theplurality of reference view images and the basic view images is removed,warping the plurality of first residual view images to a basic view, anddetermining the pruning order of the plurality of reference view imagesaccording to the number of pixels of each of the plurality of warpedfirst residual reference view images.

According to an embodiment, the acquiring of the plurality of residualreference view images by pruning the plurality of reference view imagesbased on the one or more basic view images according to the pruningorder may include acquiring a plurality of primary pruning referenceview images obtained by removing the overlapping area between theplurality of reference view images and the basic view images from theplurality of reference view images, based on the one or more basic viewimages, and acquiring the plurality of residual reference view images byremoving an overlapping area between the plurality of primary pruningreference view images according to the pruning order.

According to an embodiment, the acquiring of the plurality of residualreference view images by removing the overlapping area between theplurality of primary pruning reference view images according to thepruning order may include acquiring the plurality of residual referenceview images, by removing, from a primary pruning reference view image ofa post-order according to the pruning order, an overlapping area betweena primary pruning reference view image of a pre-order according to thepruning order and the primary pruning reference view image of thepost-order according to the pruning order.

According to an embodiment, the encoding of the one or more basic viewimages and the plurality of residual reference view images may includepacking a plurality of distributed valid areas included in the residualreference view images into one valid area, and encoding a residualreference view image including the packed valid area and packinginformation of the residual reference view.

According to an embodiment, the encoding information may include pruningorder information indicating a pruning order of the basic view imagesand the residual reference view images.

According to an embodiment, the pruning order information may beincluded in a frame header, a frame group header and/or a video header,the pruning order information of the frame header may be applied to aframe corresponding to the frame header, the pruning order informationof the frame group header may be applied to all frames of a frame groupcorresponding to the frame group header, and the pruning orderinformation of the video header may be applied to all frames of a video.

In the present disclosure, provided is a video decoding method ofdecoding a multi-view image including one or more basic view images anda plurality of reference view images including acquiring a bitstreamincluding encoding information of one or more basic view images and aplurality of residual reference view images, decoding the one or morebasic view images and the plurality of residual reference view imagesbased on the encoding information, determining a pruning order of theplurality of reference view images, and reconstructing the plurality ofreference view images from the plurality of residual reference viewimages based on the pruning order and the one or more basic view images.

According to an embodiment, the determining of the pruning order of theplurality of reference view images may include determining the pruningorder of the plurality of reference view images according to imageindices of the plurality of reference view images, and each of the imageindices may be an acquisition order of the plurality of reference viewimages.

According to an embodiment, the determining of the pruning order of theplurality of reference view images may include determining the pruningorder of the plurality of reference view images according to cameraparameters of the plurality of reference view images, and the cameraparameters may include a position and/or an angle of a camera.

According to an embodiment, the determining of the pruning order of theplurality of reference view images may include determining the pruningorder of the plurality of reference view images according to pruningorder information indicating the pruning order of the plurality ofreference view images included in the encoding information.

According to an embodiment, the pruning order information may beincluded in a frame header, a frame group header and/or a video header,the pruning order information of the frame header may be applied to aframe corresponding to the frame header, the pruning order informationof the frame group header may be applied to all frames of a frame groupcorresponding to the frame group header, and the pruning orderinformation of the video header may be applied to all frames of a video.

According to an embodiment, the reconstructing of the plurality ofreference view images from the plurality of residual reference viewimages according to the pruning order and the one or more basic viewimages may include acquiring a plurality of primary pruning referenceview images from the plurality of residual reference view imagesaccording to the pruning order, and reconstructing the plurality ofreference view images according to the plurality of primary pruningreference view images and the one or more basic view images.

According to an embodiment, the acquiring of the plurality of primarypruning reference view images from the plurality of residual referenceview images according to the pruning order may include acquiring aprimary pruning reference view image of a post-order according to thepruning order from a residual reference view image of a post-orderaccording to the pruning order, based on an overlapping area between aresidual reference view image of a pre-order according to the pruningorder and the residual reference view image of the post-order accordingto the pruning order.

According to an embodiment, the decoding of the one or more basic viewimages and the plurality of residual reference view images based on theencoding information may include acquiring packing information of theresidual reference view images from the bitstream and acquiring aresidual reference view image including a plurality of distributed validarea from a residual reference view image packed into one valid areaaccording to the packing information.

In the present disclosure, provided is a non-transitorycomputer-readable recording medium including a bitstream in which amulti-view image including one or more basic view images and a pluralityof reference view images is encoded. The bitstream includes basic viewimage encoding information of the one or more basic view images,reference view encoding information of a plurality of residual referenceview images of the plurality of reference view images, and pruning orderinformation indicating a pruning order of the plurality of referenceview images, and the plurality of residual reference view images isacquired by pruning the plurality of reference view images based on theone or more basic view images according to the pruning order.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features and other advantages of thepresent invention will be more clearly understood from the followingdetailed description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 is a view showing view images acquired by a plurality of cameraslocated at different points of view.

FIG. 2 is a view showing a method of reducing the amount of data of areference view image by removing overlapping image data between a basicview and a reference view.

FIG. 3 is a flowchart illustrating a pruning process for removingoverlapping data between view images.

FIG. 4 is a view showing a residual view image according to a pruningorder of reference view images.

FIG. 5 is a view showing a method of determining a pruning order ofreference view images in order to improve encoding efficiency.

FIG. 6 is a view illustrating an embodiment of a method of determining apruning order according to the number of pixels of a transformedresidual view image.

FIG. 7 is a flowchart illustrating an embodiment of a video encodingmethod for encoding a multi-view image including one or more basic viewimages and a plurality of reference view images.

FIG. 8 is a flowchart illustrating an embodiment of a video decodingmethod for decoding a multi-view image including one or more basic viewimages and a plurality of reference view images.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

A variety of modifications may be made to the present invention andthere are various embodiments of the present invention, examples ofwhich will now be provided with reference to drawings and described indetail. However, the present invention is not limited thereto, althoughthe exemplary embodiments can be construed as including allmodifications, equivalents, or substitutes in a technical concept and atechnical scope of the present invention. The similar reference numeralsrefer to the same or similar functions in various aspects. In thedrawings, the shapes and dimensions of elements may be exaggerated forclarity. In the following detailed description of the present invention,references are made to the accompanying drawings that show, by way ofillustration, specific embodiments in which the invention may bepracticed. These embodiments are described in sufficient detail toenable those skilled in the art to implement the present disclosure. Itshould be understood that various embodiments of the present disclosure,although different, are not necessarily mutually exclusive. For example,specific features, structures, and characteristics described herein, inconnection with one embodiment, may be implemented within otherembodiments without departing from the spirit and scope of the presentdisclosure. In addition, it should be understood that the location orarrangement of individual elements within each disclosed embodiment maybe modified without departing from the spirit and scope of the presentdisclosure. The following detailed description is, therefore, not to betaken in a limiting sense, and the scope of the present disclosure isdefined only by the appended claims, appropriately interpreted, alongwith the full range of equivalents to what the claims claim.

Terms used in the specification, ‘first’, ‘second’, etc. can be used todescribe various components, but the components are not to be construedas being limited to the terms. The terms are only used to differentiateone component from other components. For example, the ‘first’ componentmay be named the ‘second’ component without departing from the scope ofthe present invention, and the ‘second’ component may also be similarlynamed the ‘first’ component. The term ‘and/or’ includes a combination ofa plurality of items or any one of a plurality of terms.

It will be understood that when an element is simply referred to asbeing ‘connected to’ or ‘coupled to’ another element without being‘directly connected to’ or ‘directly coupled to’ another element in thepresent description, it may be ‘directly connected to’ or ‘directlycoupled to’ another element or be connected to or coupled to anotherelement, having the other element intervening therebetween. In contrast,it should be understood that when an element is referred to as being“directly coupled” or “directly connected” to another element, there areno intervening elements present.

Furthermore, constitutional parts shown in the embodiments of thepresent invention are independently shown so as to representcharacteristic functions different from each other. Thus, it does notmean that each constitutional part is constituted in a constitutionalunit of separated hardware or software. In other words, eachconstitutional part includes each of enumerated constitutional parts forconvenience. Thus, at least two constitutional parts of eachconstitutional part may be combined to form one constitutional part orone constitutional part may be divided into a plurality ofconstitutional parts to perform each function. The embodiment where eachconstitutional part is combined and the embodiment where oneconstitutional part is divided are also included in the scope of thepresent invention, if not departing from the essence of the presentinvention.

The terms used in the present specification are merely used to describeparticular embodiments, and are not intended to limit the presentinvention. An expression used in the singular encompasses the expressionof the plural, unless it has a clearly different meaning in the context.In the present specification, it is to be understood that terms such as“including”, “having”, etc. are intended to indicate the existence ofthe features, numbers, steps, actions, elements, parts, or combinationsthereof disclosed in the specification, and are not intended to precludethe possibility that one or more other features, numbers, steps,actions, elements, parts, or combinations thereof may exist or may beadded. In other words, when a specific element is referred to as being“included”, elements other than the corresponding element are notexcluded, but additional elements may be included in embodiments of thepresent invention or the scope of the present invention.

In addition, some of constituents may not be indispensable constituentsperforming essential functions of the present invention but be selectiveconstituents improving only performance thereof. The present inventionmay be implemented by including only the indispensable constitutionalparts for implementing the essence of the present invention except theconstituents used in improving performance. The structure including onlythe indispensable constituents except the selective constituents used inimproving only performance is also included in the scope of the presentinvention.

Hereinafter, embodiments of the present invention will be described indetail with reference to the accompanying drawings. In describingexemplary embodiments of the present specification, well-known functionsor constructions will not be described in detail since they mayunnecessarily obscure the understanding of the present invention. Thesame constituent elements in the drawings are denoted by the samereference numerals, and a repeated description of the same elements willbe omitted.

FIG. 1 is a view showing view images acquired by a plurality of cameraslocated at different points of view.

According to an embodiment, at least one of view images captured by aplurality of cameras may be set as a basic view image. For example, athird view image 106 may be set as a basic view image. Alternatively,instead of the third view image 106, a first view image 102, a secondview image 104 or a fourth view image 108 may be set as a basic viewimage. In addition, two or more view images may be set as basic viewimages.

According to an embodiment, based on the basic view image, the viewimages captured at different points of view may be set as reference viewimages. For example, when the third view image 106 is set as the basicview image, the first view image 102, the second view image 104 and thefourth view image 108 may be set as the reference view images.

According to an embodiment, a virtual view image may be generated bysynthesizing a plurality of view images at different points of view. Forexample, the second view image 104 may be a virtual view image generatedby synthesizing the first view image 102 and the third view image 106.In this case, an occluded area which is not visible in the third viewimage 106 may be visible in the second view image 104. Since a portionof an object corresponding to the occluded area of the third view image106 is visible in the first view image 102, the occluded area of thesecond view image 104 which is the virtual view image may bereconstructed by referring to the first view image 102.

FIG. 2 is a view showing a method of reducing the amount of data of areference view image by removing overlapping image data between a basicview and a reference view.

In FIG. 2, it is assumed that a third view image 206 is a basic viewimage and a first view image 202, a second view image 204 and a fourthview image are reference view images. However, unlike FIG. 2, anotherview image may be set as a basic view image.

First, the depth information of the third view image 206 as the basicview image and the first view image 202, the second view image 204 andthe fourth view image 208 as the reference view images are acquired. Thedepth information indicates a distance between the camera capturing eachview and the object of the view image.

In addition, based on the depth information and a three-dimensional (3D)geometric relationship between views, the third view image 206 issubjected to 3D view warping and mapped according to the views of thereference view images 202, 204 and 208. As the result of 3D view warpingand mapping, a transformed view image of each reference view isgenerated. For example, a first transformed view image 212 may begenerated by transforming the third view image 206 based on the view ofthe first view image 202. Similarly, a second transformed view image 214may be generated by transforming the third view image 206 based on theview of the second view image 204.

A partial area of the reference view image may not be visible in thetransformed view image generated from the basic view image. For example,the third view image 206 may include an occluded area which is visiblein the other view images but is not visible in the third view image 206.Accordingly, a partial area of the first view image 202 may not bevisible in the first transformed view image 212 generated from the thirdview image 206. Similarly, a partial area of the second view image 204may not be visible in the second transformed view image 214 generatedfrom the third view image 206.

As described above, an area which is visible in the reference view imagegenerated from a central reference view but is not visible in thetransformed view image may be defined as a hole area. For example, afirst occluded area 213 which is visible in the first view image 202 butis not visible in the first transformed view image 212 may be determinedas a hole area. Similarly, a second occluded area 213 which is visiblein the second view image 204 but is not visible in the secondtransformed view image 214 may be determined as a hole area.

The remaining area other than the hole area in the transformed viewimage may be visible in the basic view image. Therefore, the amount ofdata of the reference view image may be reduced by removing, from thereference view image, an overlapping area (that is, the remaining areaother than the hole area) between the reference view image and the basicview image. Accordingly, a residual view image indicating a differencebetween the reference view image and the transformed view imagegenerated from the basic view image may be generated. As a result, theresidual view image includes only the area which may be visible only inthe reference view image.

For example, a first residual view image 222 may be generated byremoving, from the first view image 202, an area which is commonlyvisible in the first view image 202 and the first transformed view image212. Similarly, a second residual view image 224 may be generated byremoving, from the second view image 204, an area which is commonlyvisible in the second view image 204 and the second transformed viewimage 214.

According to an embodiment, an overlapping area between the referenceview image and the transformed view image may be determined, bycomparing texture information and/or depth information of pixels locatedat the same coordinates or corresponding coordinates of the referenceview image and the transformed view image. For example, a pixel at whichtexture information and depth information of the reference view imageand the transformed view image are determined to be equal and/or similarto each other may be determined as an overlapping pixel. Alternatively,the overlapping pixel may be determined only by one of the textureinformation or the depth information.

According to an embodiment, when comparing texture information and/ordepth information of pixels located in corresponding coordinates betweenthe reference view image and the transformed view image, images based ona 3D spatial coordinate system that is generated through an unprojectionprocess may be used instead of images based on an image coordinatesystem that is generated through an unprojection and reprojectionprocess. Here, unprojection means converting an image of a 2D imagecoordinate system into a 3D spatial coordinate system, and reprojectionmeans converting an image of a 3D spatial coordinate system into animage of a 2D image coordinate system.

FIG. 2 shows an embodiment in which the basic view image and thereference view image are compared by transforming the basic view imageaccording to the view of the reference view image. However, on thecontrary, the basic view image and the reference view image are comparedby transforming the reference view image according to the view of thebasic view image. In addition, the overlapping area between the basicview image and the reference view image according to the result ofcomparison may be excluded from the reference view image.

FIG. 3 is a flowchart illustrating a pruning process for removingoverlapping data between view images. Specifically, FIG. 3 shows anembodiment of a pruning process of five view images. The pruning processis applicable to the depth component of an image. In addition, thepruning process is applicable not only to the depth component but alsoto a texture component of the image.

First, a basic view image and a reference view image are distinguishedbased on a predetermined algorithm. The predetermined algorithm may bedetermined according to at least one of the index of the camera, theposition of the camera, the rotation angle of the camera, the prioritybetween cameras or the position of a region of interest (ROI). In FIG.3, a first view image 302 and a second view image 304 are set as basicview images. In addition, a third view image 306, a fourth view image308 and a fifth view image 310 are set as reference view images. UnlikeFIG. 3, another view image may be set as a basic view image.

The basic view image is the highest priority in the pruning order, andthe pruning process is not performed for the basic view image. That is,since the first view image 302 and the second view image 304 are thebasic view images, the image information of the first view image 302 andthe second view image 304 is the highest priority in the pruning order.Thus, it may not be removed or compressed by the pruning process. In thepruning order, a non-pruning view, such as a basic view image, may bedesignated and marked separately and encoded.

However, the pruning process may be performed with respect to thereference view images. Accordingly, the overlapping area between thereference view image and the basic view image may be removed by thepruning process. Therefore, the image information of the third viewimage 306, the fourth view image 308 and the fifth view image 310 may beremoved or compressed according to the pruning process based on thefirst view image 302 and the second view image 304. The lowest priorityamong reference view images may be defined as a leaf view image. Inaddition, the highest priority among reference view images may bedefined as a root view image.

In the pruning process, the overlapping area may be determined accordingto the texture and/or depth information of the pixel. According to anembodiment, based on the depth information and the camera calibrationinformation, the reference view image may be transformed according to 3Dview warping and mapping based on the basic view. The texture and/orpixel information of the pixel of the transformed view image generatedby transforming the reference view image and the pixel of the basic viewimage may be compared. In addition, upon determining that the pixel ofthe transformed view image and the pixel of the basic view image aresubstantially the same as the result of comparison, it may be determinedthat the pixel of the reference view image corresponding to the pixel ofthe transformed view image is included in the overlapping area.

For example, a third transformed view image may be generated bytransforming the third view image 306 according to the view of the firstview image 302. In addition, the overlapping area between the first viewimage 302 and the third view image 306 may be determined by comparingthe third transformed view image with the first view image 302. Theoverlapping area between the second view image 304 and the third viewimage 306 may be determined using the same method.

When 3D view warping and mapping based on depth information and cameracalibration information, actual depth information corresponding to allpixels of each reference viewpoint may be used. Alternatively, apredetermined number of pixels are sampled within a depth range set asmaximum depth information and minimum depth information that can beexpressed at a corresponding reference view position, and the sampledpixels are sampled. An overlapping region may be calculated in a 3Dspace reconstructed by unprojecting the sampled pixels onto a 3D worldcoordinate system.

In the pruning process, the overlapping area between the basic viewimage and the reference view area may be removed from the reference viewimage. Accordingly, a residual view image indicating only information onan object which is not included in the basic view image may be generatedfrom the reference view image. For generation of the residual viewimage, a mask map for blocking the overlapping area between thereference view image and the basic view image may be generated from thereference view image. In addition, the residual view image may begenerated by removing the overlapping area from the reference view imageaccording to the generated mask map.

For example, a third residual view image 312 of the third view image 306may be generated by excluding, from the third view image 306, theoverlapping area between the first view image 302 and the third viewimage 306 and the overlapping area between the second view image 304 andthe third view image 306. For generation of the third residual viewimage 312, a third view mask map for blocking the overlapping areabetween the third view image 306 and the first view image 302 and theoverlapping area between the third view image 306 and the second viewimage 304 may be generated from the third view image 306. In addition,the third residual view image 312 may be generated by blocking, from thethird view image 306, the overlapping area between the third view image306 and the first view image 302 and the overlapping area between thethird view image 306 and the second view image 304 according to thethird view mask map.

Similarly, a fourth residual view image 314 of the fourth view image 308may be generated. Like the third residual view image 312, a fourth viewmask map for blocking the overlapping area between the fourth view image308 and the first view image 302 and the overlapping area between thefourth view image 308 and the second view image 304 may be determined.In addition, the fourth residual view image 314 may be generated byexcluding, from the fourth view image 308, the overlapping areaindicated by the fourth view mask map.

In order to generate the residual view image, in addition to the basicview image, an overlapping area with the other reference view image maybe further considered. Therefore, the amount of data of the residualview image may be further reduced, by further excluding, from thereference view image, the overlapping area between the reference viewimages in addition to the overlapping area between the basic view imageand the reference view image.

For example, the overlapping area between the fourth view image 308 andthe third view image 306 may be removed from the fourth view image 308.By further considering the overlapping area between the fourth viewimage 308 and the third view image 306, the fourth residual view image314 may be further reduced. Similarly, in order to generate a fifthresidual view image 316 of the fifth view image 310, the overlappingareas between the fifth view image 310 and the first view image 302 tothe fourth view image 308 may be excluded from the fifth view image 310.

According to an embodiment, a reference view image of a pre-order, whichis warped to a post-order reference view, may be used for pruning of areference view image of a post-order. The camera parameters of thereference view of the pre-order and the reference view of the post-orderare applicable to the warping process. The camera parameters may includethe capturing position and/or the capturing angle of the camera. Thepriority of the reference view and the reference view image mean thepruning order of the reference view image.

The residual view image generated by the pruning process based on thebasic view image and the reference view image includes only pixelscorresponding to the occluded area which is not visible in the basicview image. Additionally, when a secondary pruning process based on areference view image different from the current reference view image isperformed, the residual view image may include pixels corresponding tothe occluded area which is not visible even in the other reference viewimages.

The secondary pruning process may be performed using the residual viewimage in which pruning is completed. For example, when the third tofifth view images 306, 308 and 310, which are the reference view images,are sequentially pruned, only the overlapping area originating from thefirst view image 302 and the second view image 304 may be considered inthe pruning process of the third view image 306. In addition, in thepruning process of the fourth view image 308, the overlapping area fromthe third view image 306 may also be considered in addition to the firstview image 302 and the second view image 304. In addition, in thepruning process of the fifth view image 310, the overlapping areaoriginating from the third view image 306 and the fourth view image 308may also be considered in addition to the first view image 302 and thesecond view image 304.

The compression efficiency of the reference view images may varyaccording to the pruning order of the reference view image. As theoverlapping area according to the reference view image of the pre-orderis removed from the reference view image of the post-order, the encodingrate of the reference image of the post-order may be higher than that ofthe reference image of the pre-order. The pruning order of the referenceview image may be determined based on a camera index or a view imageacquisition order. Alternatively, the pruning order of the referenceview image may be determined based on the image index or view index ofeach view image.

FIG. 4 is a view showing a residual view image according to a pruningorder of reference view images.

In FIG. 4, it is assumed that a third view image 406 and a fourth viewimage 408 are basic view images and a first view image 400 and a secondview image 404 are reference view images.

A first transformed view image 412 may be generated by warping the thirdview image 406 and the fourth view image 408 according to the depthinformation of the first view image 402. In addition, a secondtransformed view image 414 may be generated by warping the third viewimage 406 and the fourth view image 408 according to the depthinformation of the second view image 404. By comparing the first viewimage 402 with the first transformed view image 412, the overlappingarea and the hole area of the first view image 402 and the third andfourth view images 406 and 408 may be derived. Similarly, by comparingthe second view image 404 with the second transformed view image 414,the overlapping area and the hole area of the second view image 404 andthe third and fourth view images 406 and 408 may be derived. In FIG. 4,the hole areas refer to an area which is visible in the first view image402 but is not visible in the first transformed view image 412 and anarea which is visible in the second view image 404 but is not visible inthe second transformed view image 414.

By removing, from the first view image 40, the overlapping area betweenthe first view image 402 and the first transformed view image 412, afirst residual view image 422 is generated. In addition, by removing,from the second view image 404, the overlapping area between the secondview image 404 and the second transformed view image 414, a secondresidual view image 424 is generated. The first residual view image 422and the second residual view image 424 include pixels corresponding tothe occluded areas of the first view image 402 and the second view image404.

A secondary pruning process may be performed with respect to the firstresidual view image 422 and the second residual view image 424. When theoverlapping area is removed in the second pruning process, similarly tothe case where the overlapping area is removed in the primary pruningprocess, warping images of the first residual view image 422 or thesecond residual view image 424 may be used.

When the pruning order of the first view image 402 is faster than thatof the second view image 404, a fourth residual view image 434 may begenerated by removing, from the second residual view image 424, theoverlapping area between the first residual view image 422 and thesecond residual view image 424. In addition, additional pruning of thefirst residual view image 422 is not performed. Accordingly, the firstresidual view image 422 and the fourth residual view image 434 becomefinal results according to the pruning process.

In contrast, when the pruning order of the second view image 404 isfaster than that of the first view image 402, a third residual viewimage 432 may be generated by removing, from the first residual viewimage 422, the first residual view image 422 and the second residualview image 424. In addition, additional pruning of the second residualview image 424 is not performed. Accordingly, the second residual viewimage 424 and the third residual view image 432 become final resultsaccording to the pruning process.

When the third residual view image 432 and the fourth residual viewimage 434 of FIG. 4 are compared, it can be seen that the amount of dataincluded in the residual view image varies according to the pruningorder. According to FIG. 4, the first residual view image 422 includesall data of the second residual view image 424. Therefore, it can beseen that there is no data in the fourth residual view image 434generated by excluding the overlapping area with the first residual viewimage 422 from the second residual view image 424. In contrast, it canbe seen that data remains in the third residual view image 432 generatedby excluding the overlapping area with the second residual view image424 from the first residual view image 422.

When the pruning order of the first view image 402 is faster than thatof the second view image 404, the hole areas of one object may bedistributed only in the first residual view image 422 between the firstresidual view image 422 and the fourth residual view image 434. Incontrast, when the pruning order of the second view image 404 is fasterthan that of the first view image 402, the hole areas of one object maybe divided into and distributed in the second residual view image 424and the third residual view image 432.

When the residual view images generated as the result of pruning arepacked in units of images, when information on one object isconcentratedly distributed in one residual view image, compressionefficiency and image quality are more advantageous as compared to thecase where the information on one object is divided into and distributedin a plurality of residual view images. Therefore, considering thatthere is no data of the fourth residual view image 434, the amount ofdata according to the first residual view image 422 and the fourthresidual view image 434 is likely to be less than the amount of dataaccording to the second residual view image 424 and the third residualview image 432. Accordingly, when the pruning order of the first viewimage 402 is faster than that of the second view image 404, encodingefficiency may increase. Therefore, it is possible to increase encodingefficiency, by concentrating data on one residual view image.

FIG. 5 is a view showing a method of determining a pruning order ofreference view images in order to improve encoding efficiency.

According to a first pruning order 500, reference view images v2, v3, v4and v5 are pruned in order of v2, v3, v4 and v5. Accordingly, thereference view image v2 is pruned based on the basic view images v0 andv1. In addition, the reference view image v3 is pruned based on thebasic view images v0 and v1 and the reference view image v2. Inaddition, the reference view image v4 is pruned based on the basic viewimages v0 and v1 and the reference view images v2 and v3. In addition,the reference view image v5 is pruned based on the basic view images v0and v1 and the reference view images v2, v3 and v4.

As described above, encoding efficiency may increase by optimizing thepruning order. Therefore, the pruning order of the reference view imagesv2, v3, v4 and v5 may be changed based on a predetermined criterion. Thepruning order may be determined based on at least one of the size of theoverlapping area between the reference view image and the basic viewimage, the size of the hole area present in the reference view image oran image acquisition order (or an image index).

According to an embodiment, the pruning order may be determined based ona ratio of the number of overlapping pixels (that is, the size of theoverlapping area) included in the overlapping area between the referenceview image and the basic view image to all pixels. It is determined thata reference view image having a small overlapping area has relativelymore information which is not visible in the basic view image.Accordingly, a high priority may be given to a reference view imagehaving a small overlapping area. Here, the high priority means that thepruning order is fast. That is, the pruning process may bepreferentially performed with respect to the reference view image havingthe small overlapping area. Accordingly, in the secondary pruningprocess of a reference view image having a large overlapping area, areference view image having a relatively small overlapping area may beused.

When the number of basic view images is two or more, the pruning ordermay be determined based on the average value, sum, maximum value orminimum value of the sizes of the overlapping areas of the basic viewimages. For example, the priority of the reference view image may bedetermined based on the sum of the sizes of the overlapping area withthe first basic view image and the overlapping area with the secondbasic view image.

According to an embodiment, the pruning order may be determinedaccording to the image acquisition order (or the image index). Forexample, the pruning order may be determined by inverting the imageacquisition order of the reference view images. Accordingly, when thefirst reference view image and the second reference view image aresequentially acquired, the pruning order of the second reference viewimage may be set to precede the pruning order of the first referenceview image.

According to an embodiment, the pruning order may be determinedaccording to a difference in camera parameter between the basic viewimage and the reference view image. The camera parameter indicates theposition and direction of the camera view. For example, when thedifference in camera view and/or direction between the reference viewimage and the basic view image is large, the reference view image may beset to have a high priority in the pruning order. In contrast, when thedifference in camera view and/or direction between the reference viewimage and the basic view image is small, the reference view image may beset to have a low priority in the pruning order.

According to an embodiment, the pruning order may be determinedaccording to the transformed residual view images warped to the basicview. For example, the pruning order may be determined according to thenumber of pixels of the transformed residual view image warped to thebasic view. When the number of pixels of the transformed residual viewimage is large, the residual view image corresponding to the transformedresidual view image may be determined to have a high priority insecondary pruning. When the number of pixels of the transformed residualview image is small, the residual view image corresponding to thetransformed residual view image may be determined to have a low priorityin secondary pruning.

FIG. 6 is a view illustrating an embodiment of a method of determining apruning order according to the number of pixels of a transformedresidual view image.

By warping the first residual view image 422 and the second residualview image 424 to the view of the third view image 406 of FIG. 4, thetransformed residual view image 600 of the first residual view image 422and the second residual view image 424 is determined. Occluded areas ofthe third view image 406 and the fourth view image 408, which arevisible in the first view image 402 and the second view image 404,appear in the transformed residual view image 600.

To determine the pruning order, the number of pixels derived from thefirst residual view image 422 among the pixels of the transformedresidual view image 600 is determined. In addition, the number of pixelsderived from the second residual view image 424 among the pixels of thetransformed residual view image 600 is determined. In addition, thepruning order is determined in descending order of the number of derivedpixels. Therefore, the pruning order of the first residual view image422 having a large number of derived pixels may be determined to have ahigh priority.

When the number of basic view images is two or more, the transformedresidual view image of each basic view is determined. In addition, thenumber of derived pixels is determined from the residual view imageamong the pixels of each transformed residual view image. In addition,the pruning order may be determined according to the number of derivedpixels.

For example, a first transformed residual view image and a secondtransformed residual view image are determined with respect to a firstbasic view and a second basic view. In addition, the number of pixelsderived from the first residual view image among the pixels of the firsttransformed residual view image and the number of pixels derived fromthe first residual view image among the pixels of the second transformedresidual view image are determined. In addition, the average value ofthe numbers of pixels derived from the first residual view image isdetermined. Similarly, the number of pixels derived from the secondresidual view image among the pixels of the first transformed residualview image and the number of pixels derived from the second residualview image among the pixels of the second transformed residual viewimage are determined. In addition, an average value of the numbers ofpixels derived from the second residual view images is determined. Inaddition, by comparing the average value of the numbers of pixelsderived from the first residual view image and the average value of thenumbers of pixels derived from the second residual view image, thepruning order of the first residual view image and the second residualview image may be determined.

When a plurality of basic view images is present, the comparison orderof the reference view image and the plurality of basic view images mayalso be variably determined. To this end, first, in FIG. 5, afterprimary pruning for simultaneously pruning all reference view imagesbased on the basic view image, a secondary pruning order is determinedand then secondary pruning is performed.

Images may be divided into a plurality of groups and a pruning processmay be independently performed for each group. One group may include atleast one image and one group may necessarily include one basic viewimage. For example, a reference view image included in a first group ispruned based on a basic view image and a reference view image includedin the first group, and a reference view image included in a secondgroup may be pruned based on a basic view image and a reference viewimage included in the second group.

FIG. 7 is a flowchart illustrating an embodiment of a video encodingmethod for encoding a multi-view image including one or more basic viewimages and a plurality of reference view images.

According to an embodiment, a basic viewpoint image having a highestpriority pruning order is determined.

In step 702, the pruning order of a plurality of reference view imagesis determined.

According to an embodiment, the pruning order of the plurality ofreference view images may be determined according to the size of theoverlapping area between the reference view image and the basic viewimage. The reference view image is assigned to a lower priority than thebase view image. In the pruning order, a non-pruning view, such as abasic view, can be specified and marked separately, and encoded. Thesize of the overlapping area between the reference view image and thebasic view image may be determined according to the pixels overlappingbetween the warped basic view image and the reference view imageaccording to the reference view.

According to an embodiment, the pruning order of the plurality ofreference view images may be determined according to the image indicesof the plurality of reference view images. The image index may be anacquisition order of the plurality of reference view images.

According to an embodiment, the pruning order of the plurality ofreference view images is determined according to the camera parametersof the plurality of reference view images. The camera parameters includethe position and/or angle of the camera.

According to an embodiment, a plurality of first residual reference viewimages, from which the overlapping areas between the plurality ofreference view image and the basic view image are removed, is acquiredfrom the plurality of reference view images. In addition, the pruningorder of the plurality of reference view images may be determinedaccording to the number of pixels of each of the plurality of firstresidual reference views according to the basic view.

In step 704, a plurality of residual reference view images is acquiredby pruning the plurality of reference view images based on one or morebasic view images according to the pruning order.

According to an embodiment, a plurality of primary pruning referenceview images, from which the overlapping areas between the plurality ofreference view images and the basic view image are removed, may beacquired from the plurality of reference view images, based on one ormore basic view images. In addition, by removing the overlapping areasbetween the plurality of primary pruning reference view images accordingto the pruning order, the plurality of residual reference view imagesmay be acquired. Specifically, by removing, from the primary pruningreference view image of the post-order according to the pruning order,the overlapping areas between the primary pruning reference view imageof the pre-order according to the pruning order and the primary pruningreference view of the post-order according to the pruning order, theplurality of residual reference view images may be acquired.

In step 706, one or more basic view images and the plurality of residualreference view images are encoded.

In step 708, a bitstream including encoding information of one or morebasic view images and the plurality of residual reference view images isoutput.

According to an embodiment, a plurality of distributed valid areas maybe included in the residual reference view image. Accordingly, theplurality of distributed valid areas may be packed into one valid area.For packing of the valid areas, the plurality of valid areas may beresized, rotated and/or moved. Packing information indicates how theplurality of valid areas has been resized, rotated and/or moved.

the plurality of valid areas of multiple residual view images may bepacked into one image. Packing information and source information ofeach valid area may be encoded. The source information may be an indexidentifying a reference view image from which a valid area is derived.

In addition, the residual reference view image including one packedvalid area and packing information of the residual reference view may beencoded. The valid area is defined as an area including information notoverlapping the basic view image in the reference view area. Since onepacked valid area is less than the whole area of the image, encodingefficiency is improved according to packing of the image.

According to an embodiment, the encoding information may includeinformation regarding pruning order.

According to an embodiment, Information indicating the number of baseview images or information for specifying the base view image may beencoded. For example, information specifying the index of the base viewimage may be encoded.

According to an embodiment, Information indicating the pruning orderbetween viewpoint images may be encoded. A viewpoint image having a lowpruning order may be reconstructed by referring to a viewpoint imagehaving a high pruning order. The pruning order of the base view image isset to take precedence over the pruning order of the reference viewimage. Therefore, the reference view image is arranged in a lower orderthan the basic view image. In the pruning order, a non-pruning view,such as a basic view image, can be distinguished from a reference viewimage and displayed as separate information.

According to an embodiment, Information about the pruning group may beencoded. At least one of information indicating the number of pruninggroups or information specifying the pruning group to which theviewpoint image belongs may be encoded. The viewpoint image may bereconstructed with reference to another viewpoint image belonging to thesame pruning group as the viewpoint image.

According to an embodiment, Information about the pruning pattern may beencoded. The pruning pattern represents a one-dimensional pattern or atwo-dimensional pattern. The one-dimensional pattern indicates that upto one other reference viewpoint image is available when pruning thereference viewpoint image. The two-dimensional pattern indicates that aplurality of different reference viewpoint images are available whenpruning the reference viewpoint images.

According to an embodiment, Information indicating whether the viewpointimage is a leaf viewpoint image may be encoded. When the viewpoint imageis a leaf viewpoint image, the corresponding viewpoint image is not usedfor pruning another viewpoint image. On the other hand, when itindicates that the viewpoint image is not a leaf viewpoint image, theviewpoint image may be used for pruning another viewpoint image. Whenthe viewpoint image is not a leaf viewpoint image, information on thenumber of next-order viewpoint images and the identifier of thenext-rank viewpoint images may for pruning may be encoded.

According to an embodiment, Information indicating whether the viewpointimage is a root viewpoint image may be encoded. When the viewpoint imageis a root viewpoint image, the corresponding viewpoint image is notpruned. On the other hand, when it indicates that the viewpoint image isnot the root viewpoint image, the corresponding viewpoint image may bepruned according to another viewpoint image. When the viewpoint image isnot the root viewpoint image, information on the number of priorityviewpoint images for pruning the viewpoint image and information on anidentifier of the priority viewpoint image may be encoded.

The coding information may be included in a frame header, a frame groupheader, an adaptation parameter set and/or a video header.

According to an embodiment, the pruning order information of the frameheader is applicable to a frame corresponding to the frame header. Thepruning order information of the frame group header is applicable to allframes of a frame group corresponding to the frame group header. Thepruning order information of the adaptation parameter set is applicableto all pictures or slices referring to the adaptation parameter set. Inaddition, the pruning order information of the video header isapplicable to all frames of video.

Alternatively, the encoding information may be encoded as separatemetadata. The video encoding method of FIG. 7 may be implemented in theform of a program instruction and stored in one or more memory devices.In addition, the program instruction in which the video encoding methodis implemented may be performed by one or more processors. In addition,the bitstream generated according to the video encoding method may bestored and distributed in one or more memory devices. In addition, thebitstream generated according to the video encoding method may betransmitted to another client device through communication, therebybeing distributed.

FIG. 8 is a flowchart illustrating an embodiment of a video decodingmethod for decoding a multi-view image including one or more basic viewimages and a plurality of reference view images.

In step 802, a bitstream including encoding information of one or morebasic view images and a plurality of residual reference view images isacquired.

In step 804, one or more basic view images and one or more residualreference view images are decoded based on encoding information.

According to an embodiment, the encoding information may include packinginformation and source information of the residual reference view. Thepacking information indicates a method of packing a plurality ofdistributed valid areas into one valid area in the encoding process. Thesource information specifies a reference view image from which a packedvalid area is derived. According to the packing information and sourceinformation, the residual reference view image including the pluralityof distributed valid areas may be acquired from the residual referenceview image packed into one valid area.

In step 806, the pruning order of one or more reference view images isdetermined.

According to an embodiment, the pruning order information indicating thepruning order of the plurality of reference view images included in theencoding information may be acquired. In addition, the pruning order ofthe plurality of reference view images may be determined according tothe pruning order information.

The pruning order information may be included in a frame header, a framegroup header, an adaptation parameter set and/or a video header. Thepruning order of the current frame may be determined by combining thepruning order information of the frame header, the frame group header,the adaptation parameter set and/or the video header. Alternatively, thepruning order information may be signaled as separate metadata.

According to an embodiment, the pruning order of the plurality ofreference view images may be determined according to the image indicesof the plurality of reference view images.

According to an embodiment, the pruning order of the plurality ofreference view images may be determined according to the cameraparameters of the plurality of reference view images. The cameraparameters may include the position and/or the angle of the camera.

In step 808, the plurality of reference view images is reconstructedfrom the plurality of residual reference view images, according to thepruning order and one or more basic view images.

With reference to the pruning sequence, the reference view image may bereconstructed using a valid area of the reference view image and a basicview image. In addition, when generating an image at an arbitraryvirtual viewpoint with reference view images reconstructed through aseries of processes, a higher weight is applied to a valid area ofanother reference view image preceded by the pruning order than thereference view area in order to generate images from an arbitraryvirtual viewpoint. According to an embodiment, a plurality of primarypruning reference view images may be acquired from a plurality ofresidual reference view images according to the pruning order.Specifically, the primary pruning reference view image of the post-orderaccording to the pruning order may be acquired from the residualreference view image of the post-order according to the pruning order,based on the overlapping area between the residual reference view imageof the pre-order according to the pruning order and the residualreference view image of the post-order according to the pruning order.In addition, the plurality of reference view images may be reconstructedaccording to the plurality of primary pruning reference view images andone or more basic view images.

The video decoding method according to FIG. 8 may be implemented in theform of program instructions and stored in one or more memory devices.In addition, the program instructions in which the video decoding methodis implemented may be performed by one or more processors.

The present disclosure provides a method of determining an optimalpruning order of a plurality of reference view images. According to theoptimal pruning order, the reference view images are pruned, therebyimproving the encoding efficiency of the reference view images.

In the above-described embodiments, the methods are described based onthe flowcharts with a series of steps or units, but the presentinvention is not limited to the order of the steps, and rather, somesteps may be performed simultaneously or in different order with othersteps. In addition, it should be appreciated by one of ordinary skill inthe art that the steps in the flowcharts do not exclude each other andthat other steps may be added to the flowcharts or some of the steps maybe deleted from the flowcharts without influencing the scope of thepresent invention.

The embodiments include various aspects of examples. All possiblecombinations for various aspects may not be described, but those skilledin the art will be able to recognize different combinations.Accordingly, the present invention may include all replacements,modifications, and changes within the scope of the claims.

The embodiments of the present invention may be implemented in a form ofprogram instructions, which are executable by various computercomponents, and recorded in a computer-readable recording medium. Thecomputer-readable recording medium may include stand-alone or acombination of program instructions, data files, data structures, etc.The program instructions recorded in the computer-readable recordingmedium may be specially designed and constructed for the presentinvention, or well-known to a person of ordinary skilled in computersoftware technology field. Examples of the computer-readable recordingmedium include magnetic recording media such as hard disks, floppydisks, and magnetic tapes; optical data storage media such as CD-RMs orDVD-RCMs; magneto-optimum media such as floptical disks; and hardwaredevices, such as read-only memory (RCM), random-access memory (RAM),flash memory, etc., which are particularly structured to store andimplement the program instruction. Examples of the program instructionsinclude not only a machine language code formatted by a compiler butalso a high level language code that may be implemented by a computerusing an interpreter. The hardware devices may be configured to beoperated by one or more software modules or vice versa to conduct theprocesses according to the present invention.

Although the present invention has been described in terms of specificitems such as detailed elements as well as the limited embodiments andthe drawings, they are only provided to help more general understandingof the invention, and the present invention is not limited to the aboveembodiments. It will be appreciated by those skilled in the art to whichthe present invention pertains that various modifications and changesmay be made from the above description.

Therefore, the spirit of the present invention shall not be limited tothe above-described embodiments, and the entire scope of the appendedclaims and their equivalents will fall within the scope and spirit ofthe invention.

What is claimed is:
 1. A video encoding method of encoding a multi-viewimage including one or more basic view images and a plurality ofreference view images, the video encoding method comprising: determininga pruning order of the plurality of reference view images; acquiring aplurality of residual reference view images, by pruning the plurality ofreference view images based on the one or more basic view imagesaccording to the pruning order; encoding the one or more basic viewimages and the plurality of residual reference view images; andoutputting a bitstream including encoding information of the one or morebasic view images and the plurality of residual reference view images.2. The video encoding method according to claim 1, wherein thedetermining of the pruning order of the plurality of reference viewimages comprises determining the pruning order of the plurality ofreference view images according to a size of an overlapping area betweena reference view image and a basic view image.
 3. The video encodingmethod according to claim 2, wherein the size of the overlapping areabetween the reference view image and the basic view image is determinedaccording to overlapping pixels between the reference view image and awarped basic view image obtained by warping the basic view image to thereference view.
 3. The video encoding method according to claim 2,wherein the size of the overlapping area between the reference viewimage and the basic view image is determined according to overlappingpixels, between the reference view image and a warped basic view imageobtained by warping the basic view image to the reference view, or awarped reference view image and the basic view image obtained by warpingthe reference view image to the basic view.
 4. The video encoding methodaccording to claim 1, wherein the determining of the pruning order ofthe plurality of reference view images comprises determining the pruningorder of the plurality of reference view images according to imageindices of the plurality of reference view images, and wherein each ofthe image indices is an acquisition order of the plurality of referenceview images.
 5. The video encoding method according to claim 1, whereinthe determining of the pruning order of the plurality of reference viewimages comprises determining the pruning order of the plurality ofreference view images according to camera parameters of the plurality ofreference view images, and wherein the camera parameters comprise aposition and/or an angle of a camera.
 6. The video encoding methodaccording to claim 1, wherein the determining of the pruning order ofthe plurality of reference view images comprises: acquiring, from theplurality of reference view images, a plurality of first residualreference view images, from which an overlapping area between theplurality of reference view images and the basic view images is removed;warping the plurality of first residual view images to a basic view; anddetermining the pruning order of the plurality of reference view imagesaccording to the number of pixels of each of the plurality of warpedfirst residual reference view images.
 7. The video encoding methodaccording to claim 1, wherein the acquiring of the plurality of residualreference view images by pruning the plurality of reference view imagesbased on the one or more basic view images according to the pruningorder comprises: acquiring a plurality of primary pruning reference viewimages obtained by removing the overlapping area between the pluralityof reference view images and the basic view images from the plurality ofreference view images, based on the one or more basic view images; andacquiring the plurality of residual reference view images by removing anoverlapping area between the plurality of primary pruning reference viewimages according to the pruning order.
 8. The video encoding methodaccording to claim 7, wherein the acquiring of the plurality of residualreference view images by removing the overlapping area between theplurality of primary pruning reference view images according to thepruning order comprises: acquiring the plurality of residual referenceview images, by removing, from a primary pruning reference view image ofa post-order according to the pruning order, an overlapping area betweena primary pruning reference view image of a pre-order according to thepruning order and the primary pruning reference view image of thepost-order according to the pruning order.
 9. The video encoding methodaccording to claim 1, wherein the encoding of the one or more basic viewimages and the plurality of residual reference view images comprises:packing a plurality of distributed valid areas included in the residualreference view images into one valid area; and encoding a residualreference view image including the packed valid area and packinginformation of the residual reference view.
 10. The video encodingmethod according to claim 1, wherein the encoding information comprisespruning order information indicating a pruning order of the basic viewimages and the residual reference view images.
 11. The video encodingmethod according to claim 10, wherein the pruning order information isincluded in a frame header, a frame group header and/or a video header,wherein the pruning order information of the frame header is applied toa frame corresponding to the frame header, wherein the pruning orderinformation of the frame group header is applied to all frames of aframe group corresponding to the frame group header, and wherein thepruning order information of the video header is applied to all framesof a video.
 12. A video decoding method of decoding a multi-view imageincluding one or more basic view images and a plurality of referenceview images, the video decoding method comprising: acquiring a bitstreamincluding encoding information of one or more basic view images and aplurality of residual reference view images; decoding the one or morebasic view images and the plurality of residual reference view imagesbased on the encoding information; determining a pruning order of theplurality of reference view images; and reconstructing the plurality ofreference view images from the plurality of residual reference viewimages based on the pruning order and the one or more basic view images.13. The video decoding method according to claim 12, wherein thedetermining of the pruning order of the plurality of reference viewimages comprises determining the pruning order of the plurality ofreference view images according to image indices of the plurality ofreference view images, and wherein each of the image indices is anacquisition order of the plurality of reference view images.
 14. Thevideo decoding method according to claim 12, wherein the determining ofthe pruning order of the plurality of reference view images comprisesdetermining the pruning order of the plurality of reference view imagesaccording to camera parameters of the plurality of reference viewimages, and wherein the camera parameters comprise a position and/or anangle of a camera.
 15. The video decoding method according to claim 12,wherein the determining of the pruning order of the plurality ofreference view images comprises determining the pruning order of theplurality of reference view images according to pruning orderinformation indicating the pruning order of the plurality of referenceview images included in the encoding information.
 16. The video decodingmethod according to claim 15, wherein the pruning order information isincluded in a frame header, a frame group header and/or a video header,wherein the pruning order information of the frame header is applied toa frame corresponding to the frame header, wherein the pruning orderinformation of the frame group header is applied to all frames of aframe group corresponding to the frame group header, and wherein thepruning order information of the video header is applied to all framesof a video.
 17. The video decoding method according to claim 12, whereinthe reconstructing of the plurality of reference view images from theplurality of residual reference view images according to the pruningorder and the one or more basic view images comprises: acquiring aplurality of primary pruning reference view images from the plurality ofresidual reference view images according to the pruning order; andreconstructing the plurality of reference view images according to theplurality of primary pruning reference view images and the one or morebasic view images.
 18. The video decoding method according to claim 17,wherein the acquiring of the plurality of primary pruning reference viewimages from the plurality of residual reference view images according tothe pruning order comprises acquiring a primary pruning reference viewimage of a post-order according to the pruning order from a residualreference view image of a post-order according to the pruning order,based on an overlapping area between a residual reference view image ofa pre-order according to the pruning order and the residual referenceview image of the post-order according to the pruning order.
 19. Thevideo decoding method according to claim 12, wherein the decoding of theone or more basic view images and the plurality of residual referenceview images based on the encoding information comprises: acquiringpacking information of the residual reference view images from thebitstream; and acquiring a residual reference view image including aplurality of distributed valid area from a residual reference view imagepacked into one valid area according to the packing information.
 20. Anon-transitory computer-readable recording medium including a bitstreamin which a multi-view image including one or more basic view images anda plurality of reference view images is encoded, the bitstreamcomprising: basic view image encoding information of the one or morebasic view images; reference view encoding information of a plurality ofresidual reference view images of the plurality of reference viewimages; and pruning order information indicating a pruning order of theplurality of reference view images, wherein the plurality of residualreference view images is acquired by pruning the plurality of referenceview images based on the one or more basic view images according to thepruning order.